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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  is  conducting 
research  to  develop  improved  methods  for  measuring  the  performance  of  commanders  and  their 
staffs  at  the  corps,  division,  and  brigade  levels.  The  materials  provided  here  were  developed  as  part 
of  the  second  phase  in  the  development  of  the  Army  Command  and  Control  Evaluation  System 
(ACCES).  This  document  is  intended  to  be  the  basic  resource  for  anyone  attempting  to  use 
ACCES  to  measure  command  and  control  performance  during  free  play  command  postexercises. 
The  ACCES  system  was  developed  for  assessing  the  performance  of  division  command  posts  but 
has  also  been  applied  at  the  corps  level. 

There  was  general  agreement  that  the  first-phase  ACCES  approach  provided  valuable 
evaluative  and  diagnostic  information  in  support  of  command  and  staff  training.  In  this  second- 
phase  effort,  ACCES  measures  were  modified  to  bring  them  more  in  line  with  doctrinal  tasks  and 
standards  and  to  refine  the  data  collection  and  analysis  procedures  to  proved  more  accurate  and 
complete  feedback  during  and  after  a  several-day  training  exercise. 

This  research  was  conducted  as  part  of  Research  Task  1122,  Battle  Command:  Improving 
Commander  and  Staff  Effectiveness,  in  coordination  with  the  Command  and  Control  Directorate 
of  the  Combined  Arms  Command-Combat  Development  (CAC-CD),  and  the  Training  and 
Doctrine  Command  (TRADOC)  System  Manager,  for  Army  Command  and  Control  Systems  (TSM 
ACCS).  COL  Dials,  TSM  ACCS,  formally  requested,  20  April  1994,  that  the  Operational  Test  and 
Evaluation  Command  (OPTEC)  apply  ACCES  to  the  evaluation  of  the  Army  Tactical  Command 
and  Control  System  (ATCCS).  Budgeting  considerations  and  planning  lead-time  precluded  the 
requested  application  in  1994,  but  OPTEC  has  stated  its  intention  to  explore  future  applications  of 
ACCES.  A  reduced  version  of  ACCES  was  used  to  generate  insights  on  the  impact  of  MOBILE 
Strike  Force  technologies  during  Prairie  Warrior  ‘94. 


ZITA  M.  SIMUTIS 
Deputy  Director 
(Science  and  Technology) 


EDGAR  M.  JOHNSON 
Director 
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THE  ARMY  COMMAND  AND  CONTROL  EVALUATION  SYSTEM  (ACCES) 

Overview 


Introduction 

This  report  provides  an  overview  of  the  Army  Command  and  Control  Evaluation  System 
(ACCES)  theory  and  methodology.  ACCES  was  initially  developed  by  Defense  Systems,  Inc. 
(DSI)  in  the  period  October,  1986  to  January,  1990  under  the  direction  of  the  Fort  Leavenworth, 
Kansas,  Research  Unit  of  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI).  In  the  spring  of  1990,  ARI  awarded  two  follow-up  contracts:  one  to  Quantum 
Research  International  (QRI)  for  support  in  conducting  ACCES  applications  and  the  second  to 
Evidence  Based  Research,  Inc.  (EBR)  for  specific  required  enhancements  to  the  ACCES  system. 
Both  contracts  expired  near  the  end  of  1993.  This  report  provides  a  description  of  ACCES 
Version  93,  and  is  current  as  of  the  end  of  December,  1993.  No  further  modification  of  the 
ACCES  methodology  is  planned  at  this  time. 

The  ACCES  methodology  largely  predates  a  major  paradigm  shift  which  occured  in  the  Army 
in  the  summer  of  1993:  the  shift  in  emphasis  to  Battle  Command  rather  than  command  and 
control  (C2).  However,  the  ACCES  concepts  in  many  ways  anticipated  that  shift;  in  particular, 
the  ACCES  model  of  C2  discussed  below  defines  the  role  of  a  commander  in  the  C2  system  in 
a  way  which  is  clearly  consistent  with  Battle  Command  concepts.  The  measures  of  effectiveness 
obtained  with  the  ACCES  system  remain  valid  measures  of  the  C2  systems  which  support  battle 
command.  For  the  sake  of  historical  continuity  we  have  choosen  not  to  rename  either  ACCES 
itself  or  any  of  the  measures. 


Background 

MACOM  commanders  and  the  Training  and  Doctrine  Command  (TRADOC),  through  the  Battle 
Command  Training  Program  and  the  Support  to  Exercise  Program,  provide  frequent  opportuni¬ 
ties  for  brigade,  division,  and  corps  staffs  to  train  in  Command  Post  Exercises  (CPXs)  and  Field 
Training  Exercises  (FTXs).  The  commanders  and  staffs  are  exposed  to  varying  environments 
and  situations  and  are  given  the  opportunity  to  practice  and  hone  their  ability  to  function  as 
effective  command  and  control  (C2 )  systems.  The  Operational  Test  and  Evaluation  Command 
(OPTEC)  is  frequently  called  on  to  evaluate  C2  materiel  systems.  Less  formal  system 
evaluations  are  conducted  throughout  C2  systems'  life-cycles  by  agencies  such  as  the  Battle 
Command  Battle  Laboratory,  Fort  Leavenworth,  Kansas,  and  the  ATCCS  Experimentation  Site, 
Fort  Lewis,  Washington. 

Both  training  and  test  and  evaluation  (T&E)  exercises  require  some  measure(s)  of  system 
performance.  In  the  training  environment,  exercises  provide  little  benefit  unless  participants  are 
provided  feedback  on  their  performance.  In  the  test  and  evaluation  environment,  measures  and 
criteria  of  system  success  or  failure  are  obviously  critical.  However,  while  measures  have  been 
developed  which  address  the  efficiency  of  selected  aspects  of  C2  performance  (e.g.,  throughput 
rates  for  certain  types  of  messages  within  certain  communication  systems),  there  is  no  accepted 
overall  measure  of  C2  effectiveness  (See  Crumley,  1989  for  a  complete  review  of  C2 
measurement  alternatives.)  Under  the  sponsorship  of  the  Combined  Arms  Command  -  Combat 
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Development  (CAC-CD),  ARI  took  steps  to  address  this  need  for  C2  measurement  through  the 
development  of  ACCES,  a  measurement  system  to  evaluate  the  effectiveness  of  C2  at  various 
levels.  ACCES  differs  from  traditional  force  effectiveness  measures  (e.g.,  force  loss  ratios) 
which  address  the  headquarters  primarily  in  terms  of  its  subordinates'  efforts.  In  contrast, 
ACCES  is  based  on  the  premise  that  C2  effectiveness  may  be  defined  as  the  effectiveness  of 
a  headquarters  staff;  in  this  view,  C2  effectiveness  measurement  requires  an  understanding  of 
the  processes  that  are  performed  by  the  staff  to  facilitate  the  performance  of  subordinate 
elements.  Thus,  we  need  a  means  to  measure  quantitatively  how  well  the  critical  staff  processes 
are  performed. 

The  ACCES  methodology  has  evolved  over  the  last  several  years.  During  that  period  ACCES 
has  provided  the  framework  for  data  collection  and  analysis  at  seventeen  division  training 
exercises  and  one  corps  training  exercise.  During  or  after  many  of  these  exercises  the  unit 
commanders  and  their  principal  staff  were  provided  feedback  on  their  C2  effectiveness  based 
on  the  ACCES  observations  and  measures.  While  no  one  will  claim  that  ACCES  is  the  ultimate 
system  which  captures  all  aspects  of  C2  effectiveness,  there  has  been  general  agreement  that  the 
ACCES  approach  has  provided  valuable  evaluative  and  diagnostic  information  in  support  of  C2 
training.  Follow-on  developmental  efforts  have  modified  ACCES  measures  in  order  to  bring 
them  more  in  line  with  doctrinal  tasks  and  standards  and  refined  the  data  collection  and  analysis 
procedures  in  order  to  provide  more  accurate  and  complete  feedback  during  and  after  a  several 
day  training  exercise.  OPTEC  is  exploring  the  application  of  ACCES  to  the  evaluation  of  the 
Army  Tactical  Command  and  Control  System  (ATCCS). 

The  second  section  of  this  paper  contains  an  expanded  description  of  the  ACCES  model, 
procedures,  and  measures. 

ACCES  in  a  Training  Environment 

Despite  the  word  “evaluation”  in  the  label  Army  Command  and  Control  Evaluation  System”, 
it  must  be  emphasized  that  ACCES  does  not  provide  a  rating  of  C2  effectiveness;  i.e.,  ACCES 
does  not  grade  a  division  HQ  as  having  passed  or  failed  against  some  set  of  criteria.  Rather, 
ACCES  provides  indicators  of  C2  effectiveness.  It  is  precisely  these  indicators  which  are  of 
value  to  a  commander  during  and  after  a  CPX.  A  major  strength  of  the  Army’s  Battle 
Command  Training  Program  (BCTP)  is  the  quality  and  quantity  of  feedback  provided  to  the 
training  audience  by  BCTP  personnel.  ACCES  indicators  provide  additional  detail  and  a 
different  perspective  on  C2  effectiveness  than  currently  provided  by  BCTP.  Unfortunately,  one 
of  the  strengths  of  ACCES,  the  ability  to  draw  from  behaviors  occurring  in  different  locations 
at  different  times,  also  is  a  weakness  of  the  system  when  applied  in  a  training  environment  with 
its  requirement  for  rapid  feedback.  A  major  focus  of  the  recent  ACCES  development  effort  was 
the  exploration  of  means  to  simplify  ACCES  data  collection  and  analysis  procedures  and 
techniques  so  as  to  reduce  the  time  required  to  provide  substantive  feedback  to  the  unit  being 
observed.  This  effort  was  less  than  fully  successful.  While  a  knowledgeable  ACCES  analyst 
can,  by  virtue  of  his  own  observations  and  discussions  with  ACCES  observers,  provide  unique 
insights  to  the  unit  during  the  course  of  the  exercise,  most  of  the  specific,  quantitative  ACCES 
Measures  still  cannot  be  calculated  until  several  days  after  the  end  of  the  exercise. 
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ACCES  in  a  Test  and  Evaluation  Environment 

In  the  typical  test  of  a  materiel  system,  the  goal  is  to  evaluate  the  system  against  system 
requirements  which  specify  performance  objectives  or  standards.  However,  there  are  few  if  any 
accepted  performance  objectives  or  standards  for  C2  systems.  We  believe  that  the  key  to  C2  test 
and  evaluation  is  the  use  of  a  stable  baseline  of  system  performance  data  which  can  be  used 
for  comparison.  Rather  than  defining  a  priori  standards  of  "successful”  C2  system  performance, 
we  argue  that  the  most  appropriate  evaluation  strategy  is  one  which  compares  the  new  system 
to  the  old.  ACCES  measures  are  targeted  to  critical  C2  system  performance  characteristics  and 
a  collection  of  ACCES  measures  across  several  units  can  provide  a  stable  baseline  for  such 
comparisons. 

There  are  at  least  two  types  of  C2  system  evaluation  which  could  utilize  ACCES.  One  target 
would  be  the  examination  of  the  effectiveness  of  the  overall  C2  system  in  a  test  unit  which  is 
trying  out  a  change  in  the  C2  system  itself  (e.g.,  a  unit  using  a  new  command  post  structure, 
or  one  using  a  new  computer-based  system).  A  second  target  would  be  the  examination  of  the 
effectiveness  of  the  overall  C2  system  in  a  test  unit  which  is  trying  out  some  change  in  one  of 
the  supporting  systems.  The  argument  here  is  that  while  a  sub-system  like  Intelligence  or  Fire 
Support  should  be  evaluated  in  its  own  right  to  determine  whether  it  is  reliable,  usable,  etc.,  it 
is  also  necessary  to  determine  whether  that  subsystem  provides  any  value-added  to  the  overall 
C2  system  in  the  context  of  the  overall  Army  Tactical  Command  and  Control  System  (ATCCS). 
In  both  types  of  C2  test  and  evaluation,  the  use  of  a  stable  ACCES  baseline  can  be  used  for 
comparison  to  establish  the  incremental  changes  in  C2  system  performance  during  the  test. 

The  continued  evolution  of  ACCES  has  made  it  difficult  to  establish  a  stable  baseline.  The 
ACCES  (version  93)  which  is  described  in  this  paper  is  the  last  in  a  series  stretching  back  five 
years.  Each  of  the  "versions”  of  ACCES  has  differed  in  some  significant  respects  from  prior 
and  subsequent  versions;  we  have  evolved  the  ACCES  model  of  command  and  control,  we  have 
changed  the  focus  of  our  measurement,  and  we  have  changed  our  data  collector  training 
procedures  (and  hence  have  changed  the  way  data  are  collected  and  analyzed).  These 
improvements  have  been  at  the  expense  of  creating  a  stable  baseline  of  similar  data  elements 
similarly  collected  under  similar  conditions  from  similar  units.  However,  many  of  the  major 
measures  have  remained  sufficiently  constant  to  enable  the  creation  of  a  useful  database 
comprising  ten  CPXs  of  which  eight  were  BCTP  Warfighter  exercises. 
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Army  Command  and  Control  Evaluation  System 


Example  Scenario 

To  provide  a  context  for  describing  the  elements  of  ACCES,  this  section  of  the  paper  provides 
a  sketch  of  a  typical  scenario  for  a  five-day  division-level  training  exercise.  In  such  an 
exercise,  the  division  headquarters  elements  and  the  headquarters  of  the  major  subordinate 
commands  would  be  in  command-post  mock-ups,  or  deployed  to  a  field  training  site.  They 
would  use  their  organic  communications  equipment  to  communicate  with  subordinate  elements 
represented  by  personnel  interacting  with  a  simulation  such  as  the  Corps  Battle  Simulation 
(CBS;  also  known  as  JESS)  or  First  Battle-BC.  Information  on  the  battle  being  played  within 
the  simulation  flows  from  lower  echelons  to  brigade  and  division  staffs;  information,  directives, 
and  requests  for  information  flow  from  the  headquarters  to  the  lower  echelons.  The  corps 
headquarters  and  adjacent  division  headquarters  are  represented  by  small  groups  of  personnel 
who  also  have  access  to  the  simulation. 

In  our  “typical”  scenario,  let  us  assume  that  the  Corps  has  directed  the  Division  to  protect  the 
northern  flank  of  the  Corps  for  a  seven-day  period  as  the  Corps  prepares  for  and  executes  a 
counter-attack  to  the  east.  Some  time  before  the  beginning  of  the  exercise,  the  Division 
Commander  chooses  an  aggressive  strategy  which  involves  moving  his  maneuver  elements 
rapidly  to  the  north  to  engage  the  expected  Enemy  Force  in  a  series  of  meeting  engagements, 
followed  by  occupation  of  strong  points  and  a  hasty  defense.  The  Division  Commander  further 
intends  to  launch  counter-attacks  as  appropriate  rather  than  establishing  prepared  defenses  at 
the  strong  points.  Based  on  this  concept  of  operation,  the  Division  operational  order  (OPORD) 
is  prepared  which  details  mission,  task  organization,  and  boundaries  for  the  elements  of  the 
Division.  Shortly  after  the  beginning  of  the  exercise,  a  fragmentary  order  (FRAGO)  is  issued 
with  additional  information  on  the  schedule  for  the  1st  Brigade  to  secure  river  crossings  and 
support  the  2nd  and  3rd  Brigades  as  they  pass  through.  As  the  fight  develops  over  the  next 
three  days,  several  FRAGOs  are  issued:  a)  adjusting  schedules  due  to  unexpected  early  success; 
b)  adjusting  resource  allocation;  c)  adjusting  boundaries  as  the  units  set  for  a  hasty  defense;  d) 
changing  mission  (“withdraw  and  reconstitute”);  and  e)  changing  task  organization.  By  day 
three  it  becomes  clear  that  the  Corps  no  longer  has  the  resources  to  support  the  Division’s 
aggressive  posture,  and  a  Corps  FRAGO  adjusts  the  Division’s  rear  boundary,  allowing  the 
Division  more  depth  in  which  to  defend  against  the  continuing  series  of  attacks  by  fresh  Enemy 
Forces.  By  day  five  the  Division  has  expended  roughly  70%  of  its  resources  (including  its 
original  resources  and  those  provided  by  Corps  during  the  exercise),  is  still  defending  within 
its  assigned  sector,  and  apparently  will  be  able  to  accomplish  the  original  mission  of  protecting 
the  Corps’  northern  flank. 

Neither  ACCES  nor  any  other  known  methodology  can  evaluate  the  Division  Commander’s 
basic  concept  of  operations  in  this  scenario.  For  example,  how  could  we  determine  whether 
the  unit’s  success  was  “worth”  the  expenditure  of  personnel,  equipment,  and  other  resources? 
How  could  we  determine  what  would  have  or  could  have  or  should  have  occurred  in  the 
execution  of  a  different  concept?  Analytical  wargaming  techniques  may  be  able  to  provide 
some  clues  to  the  relative  merits  of  alternative  concepts  given  identical  start-points,  but  the 
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typical  training  scenario,  like  the  real  world,  is  too  complex  to  allow  sufficient  replications  in 
sufficient  detail.  Furthermore,  we  must  keep  in  mind  that  we  are  observing  training  exercises 
The  commander  may  select  a  particular  course  of  action  suspecting  that  it  may  be  relatively 
ineffective  tactically  compared  to  a  more  conservative  approach,  but  choosing  to  challenge  his 
staff  and  subordinate  commanders  with  novel  problems  and  circumstances.  Rather  than 
subjectively  examining  the  commander  or  the  “quality”  of  his  decisions,  ACCES  objectively 
examines  the  C2  process  and  the  outcome  of  that  process  (i.e.,  plans  and  directives). 

Many  of  the  questions  which  would  come  to  mind  in  considering  staff  performance  in  our 
example  above  are  quite  straightforward.  Who  was  in  a  position  to  obtain  information  about 
the  evolving  enemy  situation?  Who  was  keeping  tabs  on  friendly  unit  status?  Did  those 
individuals  or  staff  sections  share  their  knowledge  with  one  another?  Was  the  information  they 
obtained  timely?  Accurate?  Complete?  Was  the  information  used  to  make  reasonable 
projections  to  possible  futures?  Were  the  information  and  projections  fully  presented  to 
appropriate  decision  makers?  Once  decisions  were  made,  were  subsequent  plans  and  orders 
based  on  timely,  accurate,  and  complete  information?  Were  the  plans  and  orders  consistent 
with  the  decision?  Were  they  coordinated  with  all  necessary  staff  sections,  and  with  higher, 
lower,  and  adjacent  commands  to  anticipate  and  avoid  problems  in  execution?  The  challenge 
in  attempting  to  measure  C2  performance  is  to  answer  all  of  these  questions,  and  others,  within 
a  coherent  framework  so  that  the  answers  provide  meaningful  insights  into  the  C2  process. 
ACCES  provides  a  framework  for  addressing  these  questions  through  a  model  of  the  C2 
process,  measures  of  the  process  which  are  derived  from  that  model,  and  analysis  techniques 
which  guide  a  systematic  process  of  deriving  quantitative  measures  from  observational  data. 

The  focus  of  ACCES  is  on  the  observable  outcome  of  C2  processes;  the  methods  and  techniques 
employed  are  of  secondary  interest,  and  then  only  to  the  extent  that  they  may  provide  clues  to 
identify  problems  that  may  be  affecting  the  quality  or  timeliness  of  the  outcomes.  For  example, 
a  staff  might  use  one  or  more  techniques  to  insure  that  the  order  they  develop  is  consistent  with 
the  guidance  provided  by  the  decision  maker.  These  techniques  include:  a)  effective  note  taking 
during  discussions  with  the  decision  maker;  b)  checking  with  others  who  were  present  when 
the  guidance  was  given;  c)  incorporating  the  decision  maker  into  the  discussions  during  the 
planning  and  orders-development  process;  d)  informal  or  formal  “brief  backs”  to  the  decision 
maker;  ore)  a  combination  of  these  techniques.  There  is  no  ACCES  measure  based  on 
observing  which,  if  any,  of  these  techniques  is  used.  Rather,  there  is  an  ACCES  measure  of 
the  outcome  of  the  process,  in  this  case  the  match  between  the  guidance  from  the  decision 
maker  and  the  order  issued. 

ACCES  Model  of  C2 

As  we  begin  to  examine  the  conceptual  model  of  C2  which  provides  the  context  for  other 
elements  of  ACCES,  it  is  important  to  define  what  we  include  in  our  use  of  the  term  “C2.”  We 
view  command  and  control  as  an  observable  behavioral  process.  People  command,  and  people 
control.  The  headquarters  elements  of  a  unit  (e.g.,  the  division  Main  Command  Post,  brigade 
Tactical  Command  Post,  etc.)  comprise  people  whose  primary  function  is  to  provide  the 
command  and  control  outputs  which  serve  to  structure  and  guide  the  actions  of  subordinate 
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units.  Those  people  work  within  constraints  established  by  tradition,  doctrine,  training,  and 
experience.  They  are  supported  by,  and  further  constrained  by,  various  C2  systems  which 
function  to  gather,  manipulate,  and  transmit  information  within  and  among  headquarters.  The 
ACCES  conceptual  model  of  C2  assumes  that  the  headquarters  of  a  unit,  the  groups  of  people 
who  do  command  and  control  combined  with  their  supporting  information  systems,  may 
themselves  be  viewed  as  a  “system”.  This  headquarters  “system”,  i.e.  the  overall  C2  system, 
establishes  goals  and  objectives  for  subordinate  units  (based  on  goals  and  objectives  provided 
by  the  senior  unit),  within  an  environment  characterized  by  a  great  deal  of  uncertainty.  This 
overall  C2  system  actively  obtains  information  about  the  environment,  reviews  that  information 
to  determine  whether  the  goals  and  objectives  are  achievable,  and  aperiodically  generates  new 
outputs  in  the  form  of  new  or  revised  goals  and  objectives  for  subordinate  units.  In  simplest 
terms  this  “adaptive  control  system”  seeks  to  achieve  a  balance  between  the  desired  and  actual 
state  within  its  environment  by  monitoring  the  state  of  the  environment  and  making  any 
necessary  changes  in  the  actions  of  the  elements  it  controls. 

The  overall  effectiveness  of  the  C2  system,  the  headquarters  of  a  unit,  can  be  judged  by  the 
viability  of  their  outputs,  and  the  critical  outputs  are  the  “plans”  and  related  directives  which 
establish  the  goals  and  objectives  for  subordinate  units.  Good  plans  can  be  executed  without 
need  for  modification  beyond  the  contingencies  built  into  them  and  remain  in  effect  throughout 
their  intended  life.  Alternatively,  a  headquarters  may  find  that  its  plans  (in  decreasing  order 
of  effectiveness): 

0  require  minor  adjustment  in  the  course  of  their  execution,  without  change  to  the  basic 
plan; 

0  require  execution  of  a  contingency,  significantly  different  from  the  intended  course  of 
action,  but  provided  for  in  the  initial  plan;  or 

°  require  cancellation  and  issuance  of  an  entire  new  plan. 

This  conceptual  model  of  C2  was  used  to  guide  the  development  of  measures  of  the  C2  process 
within  the  progenitor  of  ACCES,  the  Headquarters  Effectiveness  Assessment  Tool  (HEAT);  see 
Crumley,  1990,  for  a  detailed  discussion  of  HEAT  and  alternative  models  of  C2.  Measures  were 
developed  by  asking  what  observables  could  provide  some  insight  on  the  functioning  of  such 
an  adaptive  control  system.  The  focus  in  HEAT  and,  more  recently,  in  ACCES  has  been  on 
the  primary  C2  outputs  of  the  headquarters,  plans  and  directives,  and  on  the  information  which 
provides  the  headquarters  with  an  understanding  of  the  environment. 

The  overall  ACCES  measures  of  C2  system  effectiveness  address  primarily  the  extent  to  which 
plans  remain  in  effect  for  their  intended  period,  without  the  need  for  unanticipated  changes  in 
the  plan.  "Effective”  headquarters  are  those  which:  a)  develop  stable  plans  (presumably  based 
on  very  accurate  and  insightful  analyses  of  current  and  likely  future  status);  b)  issue  directives 
concerning  missions,  assets,  schedules,  and  boundaries  which  are  successfully  executed  without 
change;  and/or  c)  issue  directives  which  permit  flexible  responses  in  rapidly  developing 
situations  (contingency  planning). 
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ACCES  also  provides  diagnostic  measures  of  the  quality  of  processes  by  which  C2  system 
functions  are  performed.  An  exercise  timeline  and  its  associated  C2  cycles  are  used  in  ACCES 
as  the  framework  to  describe  the  information  transformation  processes  engaged  in  by  a  staff  and 
the  decision  maker,  from  the  acquisition  of  data  to  the  issuance  of  plans  and  directives. 
ACCES  also  looks  at  the  performance  of  individual  functional  cells  and  the  interactions  between 
the  cells.  The  general  approach,  as  illustrated  in  Figure  1,  is  built  around  the  following 
concepts: 


°  The  “environment”  within  which  this  adaptive  coping  process  is  attempting  to 
maintain  control  consists  of  subordinate  and  higher  headquarters,  plus  the  elements  of  METT-T 
(Mission,  Enemy,  Troops,  Terrain,  and  Time). 

0  The  staff  is  understood  to  engage  in  a  number  of  actions  in  order  to  support  decision  mak¬ 
ing  and  its  implementation: 

Collecting  information  through  monitoring  the  environment  and  receiving  reports, 
Synthesizing  information, 

_  Developing  and  evaluating  alternatives, 


General  ACCES  Model  of  C2  Process 


Planning  &  Coordination 


Processing  Directives 


Tracking  the  Situation 


Information  Handling 


Figure  1.  ACCES  C2  Processes 
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_  Reviewing  recommended  courses  of  action, 

-  Planning  implementation, 

-  Reporting, 

Coordinating, 

-  Inquiring  (seeking  information),  and 

-  Disseminating  information  in  messages  and  reports 

°  A  full  C2  cycle  in  ACCES  begins  with  the  receipt  of  information  about  the  environment, 
and  continues  with  evaluation  of  the  status  of  the  situation  vis  a  vis  the  current  plan,  recognition 
of  a  need  for  change(s)  in  the  plan,  exploration  of  alternatives,  a  decision  on  change(s),  and 
detailed  plan  development,  and  concludes  with  preparation  and  issuance  of  a  directive. 

0  ACCES  C2  cycles  will  not  always  involve  all  of  the  above  steps;  for  example,  the 
exploration  of  alternatives  and  detailed  plan  development  may  both  be  by-passed  under 
conditions  of  very  high  stress  (severe  time  pressure  while  trying  to  address  a  crisis)  or  very  low 
stress  (confirmation  that  it  is  time  to  execute  a  pre-planned  branch  or  sequel  to  the  current 
plan). 

°  The  actions  of  the  staff,  outlined  above,  yield  a  series  of  products  which  are  observable 

elements  of  the  ACCES  C2  cycles: 

Information  about  the  environment, 

-  An  initial  understanding  of  the  situation, 

The  estimate  of  the  situation,  including  a  set  of  alternative  actions,  their 
expected  results,  and  consequent  recommendations, 

-  Decisions  by  the  Commander  (or  in  some  cases  the  staff,  acting  for  the 
Commander), 

Inquiries  (for  information). 

Reports  that  inform  others,  including  answers  to  incoming  queries, 

-  Command  guidance,  and 

-  Plans  and  directives. 

The  concepts  upon  which  ACCES  is  built  assume  that  effective  staffs  look  ahead  in  time  and 
develop  plans  that  are  robust  (i.e.,  plans  that  will  support  mission  accomplishment  despite 
changes  in  the  elements  of  METT-T).  ACCES  v.  93  (as  shown  in  Tables  1-7)  includes  56 
primary  measures  of  performance;  of  these,  7  of  the  measures  deal  with  the  planning  process 
as  a  whole  and  the  remainder  are  descriptive  measures  which  focus  on  the  process  steps 
separately.  An  additional  107  subordinate  measures  provide  a  rich  source  of  additional 
diagnostic  information  for  an  analyst  examining  a  particular  exercise. 

Two  additional  categories  of  measures  are  also  developed  in  ACCES.  Table  8  lists  the  six 
descriptive  "measures”  of  the  decision  context:  for  example,  for  each  decision  recorded,  who 
made  the  decision?  What  unit(s)  is/are  affected?  Appendix  D  of  ACCES  Documentation 
(Hayes,  et  al.,  1995)  lists  16  primary  and  13  subordinate  descriptive  "measures”  of  the  exercise 
and  scenario;  these  elements  of  ACCES  come  into  use  as  we  aggregate  and  compare  data  across 
different  exercises. 
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Table  1 

PRIMARY  ACCES  MEASURES  -  v.  93 


GE 

NERAL  MEASURES  (Overall  planning  process) 

Item  ID  Title 

Brief  Definition 

Subordinate  Measures 

G  1.0 

Plan 

Duration 

Time  from  implementation  of  the  plan 
to  time  it  is  changed  in  some  substan¬ 
tive  way  or  completed 

4  -  Duration  of  each  of  the  four  major 
plan  elements:  Mission; 

Task  Organization;  Boundary;  and 
Schedule. 

G  2.0 

A  percentage  based  on  time:  Plan  Du¬ 
ration  (G  1.0  )  v.s'.  the  intended  life  of 
the  plan  (PD  5.0) 

4  -  Stability  of  each  of  the  4  major 
plan  elements:  Mission;  Task... 

G  3.0 

Plan 

Execution 

A  percentage  based  on  the  number  of 
all  major  plan  elements  which  are 
completed  within  original  contingen¬ 
cies,  indicating  sufficient  leeway  for 
adaptation  to  battlefield  conditions 

0  -  However,  note  that  each  of  the  4 
major  plan  elements  could  be  called  out 
separately  when  summarizing  across 
time. 

G  4.0 

Planning 

Success 

A  percentage  based  on  the  number  of 
plans  which  are  completed  without 
change  ("dominant”)  or  within 
original  contingencies  ("adaptive”) 
compared  to  the  total  number  of  plans 

0  -  However,  requires  computation  of: 
total  number  of  plans;  number  of  plans 
which  were  "dominant”  ;  number  of 
plans  which  were  "adaptive.” 

G  5.0 

Planning 

Initiative 

The  percentage  of  all  plans  which  are 
"proactive”  (assume  any  changes  in 
the  battlefield  will  be  under  own 
force’s  control)  or  "contingent”  (antici¬ 
pate  some  change  which  will  be 
handled  by  established  contingencies) 
rather  than  "reactive”  (assume  own 
force  has  little  or  no  control  over  pos¬ 
sible  changes  in  the  battlefield) 

0  -  However,  requires  computation  of: 
number  of  plans  which  were  "proac¬ 
tive”;  number  of  plans  which  were 
“contingent”. 

G  6.0 

Planning 

Cycle  Time 

The  time  from  awareness  of  need  to 
the  time  directive  is  issued 

3-  Planning  cycle  time  under  each  of 
three  conditions”  Low;  Moderate;  or 

High  “planning  stress”.  The  degree  of 
planning  stress  is  a  function  of  the 
degree  of  success  of  the  prior  plan. 

G  7.0 

C2  Impact  on 
Plans 

The  percentage  of  changes  to  plans 
that  are  not  attributable  to  the  failure 
of  C2  processes 

6  -  Impact  of  I,  SA,  CA,  PD,  IE,  and  O 
processes. 

Table  2 

PRIMARY  ACCES  MEASURES  -  v.  93 


INCOMI 

NG  INFORMATION  HANDLING  (Formal  and  Informal  Reports) 

Item  ID 

Title 

Brief  Definition 

Subordinate  Measures 

I  1.0 

Scheduled 

Situation 

Reports 

(SITREPs) 

Received 

The  number  of  Scheduled  SITREPs  received  is  tallied 
by  24  hour  period.  These  are  formal,  scheduled 
reports  on  friendly  units  that  contain:  unit  ID, 
capability,  combat  activity,  and  location. 

2  -  Percentage  received  late  and 
Degree  of  lateness 

12.0 

Scheduled 

Intelligence 

Summaries 

(INTSUMS) 

Received 

The  number  of  scheduled  INTSUMS  received  is  tallied 
by  24  hour  period.  These  are  formal,  scheduled 
reports  on  enemy  units  that  contain:  unit  ID,  capability, 
combat  activity,  and  location. 

2  -  Percentage  received  late  and 
Degree  of  lateness 

13.0 

Scheduled 

SITREP 

Elements 

Reported 

The  number  of  elements  that  are  included  in  the 
scheduled  SITREPs 

0  -  However,  note  that  the  four 
elements  could  be  tallied  sepa¬ 
rately. 

14.0 

Scheduled 

INTSUM 

elements 

reported 

The  number  of  elements  that  are  included  in  the 
scheduled  INTSUMs 

0  -  However,  note  that  these 
elements  could  be  tallied  sepa¬ 
rately. 

I  5.0 

Scheduled 

SITREP 

accuracy 

The  percentage  of  reported  non-location,  scheduled 
SITREP  elements  that  are  correct.  Elements  which 
reflect  location  accuracy  are  reported  seperately. 

4  -  Percentage  correct  for  each 
of:  Unit  ID,  capability,  and  com¬ 
bat  activity;  deviations  in  meters 
from  Ground  Truth 
for  locations. 

16.0 

Scheduled 

INTSUM 

accuracy 

The  percentage  of  reported  non-location,  scheduled 
INTSUM  elements  that  are  correct.  Elements  which 
reflect  location  accuracy  are  reported  seperately. 

4  -  Percentage  correct  for  each 
of:  Unit  ID,  capability,  and  com¬ 
bat  activity;  deviations  in  meters 
from  Ground  Truth  for  locations. 

I  7.0 

Unscheduled 

SITREP 

accuracy 

The  percentage  of  unscheduled  SITREP  non-location 
elements  which  are  correct.  Elements  which  reflect 
location  accuracy  are  reported  seperately. 

The  elements  tallied  for  the  scheduled  and  unscheduled 
SITREPs  are  the  same. 

4  -  Percentage  correct  for  each 
of:  Unit  ID,  capability,  and  com¬ 
bat  activity;  deviations  in  meters 
from  Ground  Truth  for  locations. 

I  8.0 

Spot 

Report 

Accuracy 

The  percentage  of  non-location  report  elements  which 
are  correct.  Elements  which  reflect  location  accuracy 
are  reported  seperately. 

Spot  reports  are  unscheduled,  informal  reports  of 
knowledge  of  enemy  units.  The  same  elements  are 
tallied  for  Spot  Reports  as  for  INTSUMS. 

4  -  Percentage  correct  for  each 
of:  Unit  ID,  capability,  and  com¬ 
bat  activity;  deviations  in  meters 
from  Ground  Truth  for  locations. 

19.0 

Queries  to 
Scheduled 
SITREPs 

The  percentage  of  scheduled  report  elements  which  are 
queried 

1  -  Percentage  of 
unscheduled  report  elements 
queried 

I  10.0 

Queries  to 
scheduled 
INTSUMs 

The  percentage  of  scheduled  report  elements  which  are 
queried 

1  -  Percentage  of 
unscheduled  report  elements 
queried 

Table  3 

PRIMARY  ACCES  MEASURES  -  v.  93 


SITUATION  ASSESSMENT  MEASURES 

Item  ID 

Title 

Brief  Definition 

Subordinate  Measures 

SA  1.0 

Accuracy  of 
Assessments  of 
the 

Friendly 

Situation 

The  percentage  of  SAs  about  friendly 
forces  that  turned  out  to  be  correct 

A  Situation  Assessment  conveys  an  un¬ 
derstanding  of  the  operating  environ¬ 
ment  that  goes  beyond  factual  details  to 
an  appreciation,  forecast,  or  estimate. 

4  -  distinguishing  comprehensive 
from  casual  SAs  and  counting  the 
elements  considered 

The  elements  of  a  friendly  SA  are: 
mission,  task  organization,  disposi¬ 
tion,  activities,  status,  and  support. 

SA  2.0 

Accuracy  of 
Assessments  of 
the 

Enemy 

Situation 

The  percentage  of  SAs  about  enemy 
forces  that  turned  out  to  be  correct 

4  -  distinguishing  comprehensive 
from  casual  SAs  and  counting  the 
elements  considered 

The  elements  of  an  enemy  SA  are: 
composition,  disposition,  combat 
power,  activities,  and  capabilities 
(which  includes  intentions). 

SA  3.0 

Alternative 

Futures 

The  number  of  alternative  futures 
explicitly  considered  during  SAs 

0 

SA  4.0 

Time  Span  of 
Assessments 

The  time  from  the  expression  of  an 
assessment  to  the  end  of  the  period  that 
the  assessment  covers 

0 

SA  5.0 

Assessment 
agreement 
within  CPs 

The  percentage  of  agreement  within 

CPs  (between  CP  sections)  on  situation 
assessments  (SAs)  of  friendly  and  ene¬ 
my  forces. 

2  -  separate  tallies  for  friendly  and 
enemy 

SA  6.0 

Assessment 
agreement 
between  CPs 

The  percentage  agreement  between  CPs 
on  SAs  of 

friendly  and  enemy  forces 

2  -  separate  tallies  for  friendly  and 
enemy 
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Table  4 

PRIMARY  ACCES  MEASURES  -  v.  93 


OUTGOING  INFORMATION  HANDLING  (Periodic  Reports  and  Spot  Reports) 

Item  ID 

Title 

Brief  Definition 

Subordinate  Measures 

O  1.0 

Scheduled 

Situation 

Reports 

(SITREPs) 

Sent 

The  number  of  scheduled  SITREPs 
sent,  tallied  by  24  hour  period 

2  -  Percentage  of 

scheduled  SITREPs  sent  late  and 

times  late  of  those  sent  late 

0  2.0 

Scheduled 

Enemy 

Intelligence 

Summaries 

(INTSUMs) 

Sent 

The  number  of  scheduled  INTSUMs 
sent,  tallied  by  24  hour  period 

2  -  Percentage  of  scheduled 
INTSUMs  sent  late  and  times  late 
of  those  sent  late 

0  3.0 

Scheduled 

SITREP 

Elements 

Reported 

The  number  of  scheduled  SITREP  ele¬ 
ments  included  in  reports 

0 

0  4.0 

Scheduled 

INTSUM 

Elements 

Reported 

The  number  of  scheduled  INTSUM 
elements  included  in  reports 

0 

0  5.0 

Scheduled 

SITREP 

Accuracy 

The  percentage  of  reported 
non-location,  scheduled 

SITREP  elements  that  are  correct 

4  -  1  each  for  unit  ID, 
capabilities,  combat  activities,  and 
location 

0  6.0 

Scheduled 

INTSUM 

Accuracy 

The  percentage  of  reported 
non-location,  scheduled 

INTSUM  elements  that  are  correct 

4-1  each  for  unit  ID, 
capabilities,  combat  activities,  and 
location 

0  7.0 

Unscheduled 

SITREP 

Accuracy 

Parallel  to  0  5.0 

The  focus  here  is  on  Friendly  Spot 
Reports  rather  than  the  periodic  re¬ 
ports  considered  in  0  5.0. 

4  -  Same  as  for 

0  5.0 

0  8.0 

Spot  Report 
Accuracy 

Parallel  to  0  6.0 

The  focus  here  is  on  Enemy  Spot  Re¬ 
ports  rather  than  the  periodic  reports 
considered  in  0  6.0. 

4  -  Same  as  for 

0  6.0 
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Table  5 

PRIMARY  ACCES  MEASURES  -  v.  93 


INFORM/ 

VTION  EXCHANGE  (within  and  between  CPs) 

Item  ID 

Title 

Brief  Definition 

Subordinate  Measures 

IE  1.0 

Coordination 

Cycle  Time 

Within  CPs 

The  time  between  the  recognition  of  a 
need  for  coordination  and  the  resolution  of 
the  issue 

2  -  time  from  recognition  of  need  to 
initiation  of  action  and  from  initiation 
to  resolution 

These  two  sum  to  IE  1.0. 

IE  2.0 

Coordination 
Frequency 
within  CPs 

The  number  of  recognized  circumstances 
requiring  coordination  within  a  CP 

2  -  Percentage  of  recognized 
coordinations  initiated  and  percentage 
of  initiated  coordinations  completed 

IE  3.0 

Coordination 

Cycle  Time 
between  CPs 

The  time  between  recognition  of  the  need 
for  coordination  and  the  time  an  issue  is 
resolved. 

NOTE  that  “coordination”  is  not  informa¬ 
tion  seeking,  but  rather  is  the  request  for 
substantive  comment  on,  or  input  to,  a 
plan  or  SA. 

2  -  Same  as  IE  1.0  but  between  CPs 

IE  4.0 

Coordination 
Frequency 
between  CPs 

Parallel  to  IE  2.0,  with  the  focus  here  on 
coordinations  between  different  CPs  (TAC. 
MAIN,  REAR,  and  BDE). 

2  -  Same  as  for 

IE  2.0  but  between  CPs 

IE  5.0 

Information 

Seeking 

Cycle  Time 
within  CPs 

The  time  between  the  recognition  of  a 
need  for  information  and  the  resolution  of 
the  issue 

2  -  time  from  recognition  of  need  to 
initiation  of  action  and  from  initiation 
to  resolution 

These  two  sum  to  IE  5.0. 

IE  6.0 

Information 

Seeking 

Frequency 
within  CPs 

The  number  of  recognized  circumstances 
requiring  information  seeking  within  a  CP 

2  -  Percentage  of  recognized  seekings 
initiated  and  percentage  of  initiated 
seekings  completed 

IE  7.0 

Information 

Seeking 

Cycle  Time 
between  CPs 

The  time  between  the  recognition  of  a 
need  for  information  and  the  resolution  of 
the  issue 

2  -  time  from  recognition  of  need  to 
initiation  of  action  and  from  initiation 
to  resolution 

These  two  sum  to  IE  5.0. 

IE  8.0 

Information 

Seeking 

Frequency 
between  CPs 

The  number  of  recognized  circumstances 
requiring  information  seeking  between  CPs 

2  -  Percentage  of  recognized  seekings 
initiated  and  percentage  of  initiated 
seekings  completed 
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Table  6 

PRIMARY  ACCES  MEASURES  -  v.  93 


COURSE  OF  ACTION  ANALYSES 

Item  ID 

Title 

Brief  Definition 

Subordinate  Measures 

CA  1.0 

Number  of 
Staff 

Participants 

The  median  number  of  staff 
members  actively  participating 
during  COA  Analyses 

0 

CA  2.0 

Variety  of 
Staff 

Participants 

The  median  number  of  staff 
sections  represented  during 

COA  Analyses. 

0 

CA  3.0 

Number  of 
COA’s 

The  median  number  of  COA’s 
explicitly  considered  during 

COA  Analyses. 

1  -  the  number  seriously  and 
carefully  considered 

CA  4.0 

Completeness 
of  COA 
Analyses 

The  percentage  of  COA  Anal¬ 
yses  which  included  explicit 
discussion  of  all  of  the  follow¬ 
ing:  predictions  of  enemy  reac¬ 
tions;  likelihood  of  mission  ac¬ 
complishment;  residual  capacity 
of  friendly  units;  residual  capac¬ 
ity  of  enemy  units. 

4—1  for  each  of  the  four 
elements  of  analysis  listed  in 
the  definition. 

CA  5.0 

Time  Span  of 
COA 

Analyses 

This  measure  parallels  SA  6.0 

SA  6.0  addresses  the  time  span 
of  situation  assessments  (“This 
is  what  is  happening  and  is 
likely  to  happen”),  while  COA 
5.0  addresses  the  time  span  of 
“what  if’  analyses  during  devel¬ 
opment  of  a  plan. 

0 
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*Note  that  COA  a 

inalyses  and  the  prepare 
Discussion 

PREPARATION  OF  DIRECTIVES* 

ition  of  directives  may  be  occurring  simultaneously,  and  may  involve  many  of  the  same  people, 
s  which  serve  both  purposes  are  counted  in  both  sets  of  measures. 

Item  ID 

Title 

Brief  Definition 

Subordinate  Measures 

PD  1.0 

Number  of  Staff 
Participants. 

The  number  of  staff  members  actively  partici¬ 
pating  during  preparation  of  directives. 

0 

PD  2.0 

Variety  of  Staff 
Participants 

The  number  of  staff  sections  represented  during 
preparation  of  directives. 

0 

PD  3.0 

Directive 
preparation  time 

The  time  interval  between  a  decision  and  the 
issuance  of  the  implementing  directive. 

0 

PD  4.0 

Warning  Order 

Time 

The  time  interval  between  a  decision  and  the 
issuance  of  a  warning  order 

0 

PD  5.0 

Time  Span  of  Di¬ 
rective 

The  length  of  time  that  the  directive  is  to  be 
effective 

For  example,  “On  order,  move  to  position 
xxxyyy  and  prevent  enemy  penetration  of  your 
sector  for  24  hours”  has  a  24  hour  time  span. 
Many  directives  neglect  start  time,  end  time,  or 
both,  and  cannot  be  assessed. 

1  -  Percentage  of  directives  that  are  event 
driven  (rather  than  specifying  times) 

PD  6.0 

Directive  match 
with  decision 

The  percentage  of  directive  elements  that  are 
consistent  with  the  elements  of  the  decision 

The  decision  maker’s  decision  with  respect  to 
mission,  task  organization,  boundaries,  and 
schedule  (as  stated  during  initial  guidance  or 
follow-up  discussions)  are  compared  to  the  ele¬ 
ments  of  the  directive(s)  issued. 

0 

PD  7.0 

Clarity  of 

Directives 

The  percentage  of  directives  which  do  not  re¬ 
quire  clarification  by  the  issuing  HQ. 

0 

PD  8.0 

Consistency  of 
Directives  between 
CPs 

The  percentage  of  directives  issued  by  alternate 
CPs  that  do  not  conflict  with  directives  issued 
by  the  primary  CP 

0 

PD  9.0 

Lead-Time  for 
Directive  planning 

The  time  from  receipt  of  a  directive  by  a  subor¬ 
dinate  command  to  the  time  the  first  element  of 
the  directive  is  to  be  executed. 

3  -  The  ratio  of  (PD  9.0)  to:  [the  time  the 
first  element  of  the  plan  is  to  be 
implemented  minus  the  time  from  the  first 
perception  of  need  for  a  directive];  a 
parallel  measure  of  lead  time  provided  by 
warning  orders;  and  a  similar  ratio  for 
warning  order  lead  time. 

TOTALS: 

56  Primary  Measures 


107  Subordinate  Measures 


TABLE  8 

ACCES  MEASURES  OF  DECISION  CONTEXT  -  v.  93 


DECISION  CONTEXT 

Item  ID 

Title 

Brief  Definition 

Subordinate  Measures 

DC  1.0 

Decision 

Maker 

Position  of  the  person  making  a 
given  decision 

Could  include  Commander,  a 
subordinate  in  the  commander’s 
name,  ADC(S)  or 

ADC(M),  COS,  G3/S3,  etc. 

0 

DC  2.0 

Affected 

Units 

Listing  of  subordinate  units 
affected  by  the  decision  to 
change  an  element  of  the  plan 

0 

DC  3.0 

Decision 

focus 

Listing  of  the  element(s)  of  a 
plan  affected  by  a  given  deci¬ 
sion. 

Could  include  mission,  task 
organization,  schedule,  etc. 

0 

DC  4.0 

Contingency 

Did  a  given  decision  involve 
activation  of  a  previously 
planned  contingency? 

0 

DC  5.0 

Decision 

time 

Time  at  which  a  decision  was 
made 

0 

DC  6.0 

Type  of 
operation 

Identification  of  the  type  of 
operation  affected  by  the 
decision. 

Categories  of  operations  are 
taken  from  ARTEP  71-100- 
MTP. 

0 
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Application  of  ACCES 

The  application  of  ACCES  comprises  five  stages: 

°  Prior  coordination  with  the  unit  and  "customization"  of  ACCES  procedures  and  measures 
if  required; 

°  Obtaining  and  training  data  collectors; 

°  Data  collection; 

°  Data  analysis;  and 

°  Report  preparation 

Each  of  these  stages  will  be  discussed  in  turn. 

Prior  Coordination  and  Customization.  The  key  to  ACCES  measures  is  information 
exchange  within  and  between  C2  elements.  The  ACCES  model  of  command  and  control  implies 
that  an  effective  staff: 

a.  knows  about  and  understands  the  meaning  of  the  tactical  situation  ("seeing  the  battle"); 

b.  shares  that  information  and  knowledge  with  other  elements  of  the  staff,  with  the 
commander,  and  with  higher,  lower,  and  adjacent  units  ; 

c.  uses  that  information  and  knowledge  in  conjunction  with  the  commander's  guidance  to 
develop  plans  ("planning"  and  "wargaming"); 

d.  provides  the  necessary  information  to  the  commander  to  allow  him  to  make  an  informed 
decision  on  the  preferred  course  of  action; 

e.  creates  and  communicates  operational  orders  (OPORDs)  or  fragmentary  orders 
(FRAGOs)  which  convey  to  subordinate  commanders  an  unambiguous  statement  of  the 
commander's  intent  and  guidance,  particularly  with  respect  to  mission,  assets,  boundaries,  and 
schedule. 

ACCES  can  track  and  assess  all  of  these  steps,  but  only  if  the  information  being  captured, 
shared,  and  used  is  noted  by  an  ACCES  data  collector  or  is  obtained  through  other  means.  It 
is  straightforward  to  assess  the  timeliness,  completeness,  and  accuracy  of  the  "see  the  battle" 
staff  processes  if  data  collectors  note  what  information  is  available  within  a  given  staff  element 
and  what  discussions  occur,  and  if  there  is  a  "ground  truth"  available  for  comparison  purposes. 
However,  the  data  collectors  can  never  hope  to  obtain  more  than  a  sample  of  all  information 
available  and  being  discussed  at  a  given  location;  furthermore,  there  are  obvious  practical  limits 
on  the  number  of  CP  elements  which  can  be  observed  and  on  the  number  of  observers  within 
a  given  location.  It  is  therefore  necessary  to  focus  the  observers’  attention  on  the  relevant  staff 
actions. 

The  prior  coordination  step  is  an  important  element  in  a  successful  ACCES  application  because 
it  provides  the  information  on  the  scope  and  purpose  of  the  exercise,  providing  the  proper  focus 
for  the  data  collection  efforts.  If  a  given  exercise  is  primarily  to  evaluate  a  modified  Combat 
Service  Support  (CSS)  system,  then  the  data  collectors  will  need  to  be  located  where  CSS 
information  is  handled.  Furthermore,  some  of  the  measures  which  focus  on  tactical  situation 
information  will  need  to  be  re-defined  to  incorporate  the  CSS  focus.  If  a  given  exercise 
involves  only  one  echelon  or  one  or  two  staff  elements  within  an  echelon,  then  measures  of 
staff  coordination  will  be  downplayed  or  ignored  altogether  in  the  training,  data  collection,  and 
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analysis  phases.  The  one  ACCES  failure  was  due  to  problems  in  this  phase  of  the  application. 
We  attempted  to  apply  ACCES  during  an  evaluation  of  the  All  Source  Analysis  System  (AS AS) 
but  were  unable  to  collect  any  meaningful  data.  In  this  instance,  there  was  prior  coordination, 
but  the  scope  and  form  of  the  test  was  modified  several  times  in  the  final  weeks  before  the 
exercise;  we  were  not  able  to  obtain  enough  information  early  enough  to  successfully  customize 
ACCES  for  this  atypical  exercise. 

In  our  recent  experience  with  ACCES  applications,  the  units  hosting  our  ACCES  observers  have 
all  been  involved  in  Battle  Command  Training  Program  (BCTP)  training  exercises,  known  as 
Warfighter  Exercises  (WFXs).  The  conditions  for  WFXs  and  many  division-level  CPXs 
(usually  conducted  by  the  division  with  corps  involvement)  are  quite  similar  in  terms  of  the 
number  and  types  of  CP  elements  which  are  involved.  Under  these  circumstances,  little  or  no 
special  advance  preparation  is  required  to  conduct  a  successful  ACCES  data  collection  effort. 
However,  our  past  experience  has  shown  us  that  apparently  minor  variations  from  the  norm  can 
severely  disrupt  data  collection  and  data  analysis  to  the  point  that  few  if  any  useful  data  are 
obtained,  and  few  if  any  insights  are  generated  from  the  analyses. 

Obtaining  and  Training  Data  Collectors.  ARI  took  steps  to  establish  and  maintain  a  trained 
cadre  of  ACCES  observers  (data  collectors)  standing  ready  to  go  to  an  exercise.  A  task-order 
contract  was  awarded  to  Quantum  Research  International  for  ACCES  data-collection  and 
analysis  support.  Under  this  contract  Quantum  was  committed  to  provide  up  to  eight  data 
collectors  on  any  given  exercise,  and  to  support  ARI  in  the  analysis  of  the  data  following  the 
exercise.  The  combination  of  the  eight  Quantum  data  collectors,  one  or  two  from  within  the 
Fort  Leavenworth  Field  Unit  staff,  and  one  or  two  from  other  ARI  elements  put  us  well  on  our 
way  to  finding  the  16-20  data  collectors  required  for  an  ACCES  application.  However,  it  is 
unlikely  that  any  one  agency,  including  ARI,  will  long  be  able  to  maintain  enough  experienced 
data  collectors  to  provide  more  than  a  solid  core  of  the  necessary  people.  Even  with  experi¬ 
enced  people,  some  refresher  training  is  necessary.  General  practice  was  to  obtain 
supplementary  volunteer  data  collectors  from  a  variety  of  Army  agencies  and  from  government 
contractors.  The  benefit  to  the  data  collectors  is  the  opportunity  to  see  the  C2  processes  in  an 
actual  unit,  an  invaluable  experience  for  a  young  TRADOC  staff  officer  or  civilian  contractor 
involved  in  designing  the  next-generation  C2  system.  Data  collectors  will  have  a 
wide  range  of  experience  in  tactical  C2,  with  many  having  very 
little  such  experience  at  the  division  echelon.  Thus,  training  was 
required  both  on  the  ACCES  procedures  and  on  tactical  C2.  Latest  practice  was  to  gather  the 
team  of  data  collectors  near  the  exercise  site  three  days  prior  to  the  exercise.  Two  days  are 
spent  reviewing  the  ACCES  procedures  and  other  material;  time  is  also  allowed  for  orientation 
and  coordination  visits  to  the  exercise  sites  and  for  discussions  with  BCTP  personnel.  EBR  and 
their  subcontractor,  BDM,  have  developed  training  packages  including  Programs  of  Instruction 
(POI)  and  detailed  back-up  material  for  two-day  initial  training  and  for  advanced  training  for 
ACCES  analysts.  See  ACCES  Documentation  (Hayes,  et  al.,  1995)  for  these  POIs. 

Data  Collection.  During  the  exercise  being  observed,  data  collectors  are  stationed  at 
"critical"  C2  nodes,  as  determined  by  the  scope  and  purpose  of  the  exercise.  A  given  observer 
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is  stationed  in  the  same  location  each  day,  and  he  or  she  occupies  that  position  for  roughly  l2Vi 
hours  each  day,  including  the  10-15  minutes  required  at  each  end  of  the  shift  to  hand  off  to  the 
person  on  the  alternate  shift  (assuming  a  24  hour-per-day  exercise  schedule).  The  location  may 
be  one  small  area  which  can  be  observed  almost  entirely  from  a  fixed  point  (e.g.,  the  G3-Plans 
cell  at  division  Main)  or  may  require  movement  among  different  points  in  order  to  be  able  to 
capture  the  data  (e.g.,  a  division  Tactical  CP).  Data  collectors  are  provided  with  a  clipboard 
and  data  sheets  and  are  asked  to  record  all  relevant  information  exchange.  (Figure  2  shows  a 
replica  of  the  data  collectors'  log  sheets.)  Discussions  of  the  quality  of  the  field  rations  are 
ignored,  but  receipt  of  a  status  update  from  a  maneuver  brigade,  discussions  of  the  likely  intent 
of  an  enemy  unit,  or  an  assessment  of  the  situation  provided  during  a  formal  briefing  or  a  shift- 
change  briefing  should  always  be  noted.  Periodically  the  data  collector  will  record  the  location 
and  other  information  about  critical  friendly  and  enemy  units  as  posted  on  map  overlays  or 
status  boards. 

An  additional  data  collector  is  stationed  at  the  exercise  simulation  center.  This  person  is 
responsible  for  capturing  the  simulation  “ground  truth”  for  later  use  in  assessing  the  unit’s 
timeliness  and  accuracy  of  information  handling,  their  accuracy  in  forecasting  future  battlefield 
situations,  etc.  This  observer  needs  to  also  maintain  a  clear  awareness  of  the  critical  tactical 
and  training  events  throughout  the  exercise.  Our  later  analysis  of  data  collected  by  each  CP 
observer  must  be  done  in  the  context  of  the  overall  exercise,  and  only  the  simulation-center 
observer  is  in  a  position  to  provide  that  context.  This  observer  also  obtains  descriptive  informa¬ 
tion  about  the  unit  and  exercise  as  detailed  in  ACCES  Documentation  (Hayes,  et  al.,  1995). 

Data  analysis.  The  analysis  of  the  raw  data  proceeds  in  two  steps.  The  first  step  involves 
the  "reduction”  and  "collation”  of  the  data,  as  it  was  recorded  on  the  logs  kept  by  the  data 
collectors,  onto  a  set  of  intermediate  “data-reduction  forms”.  There  are  seven  of  these  forms 
titled  in  accord  with  Tables  1-8  except  that  a  single  form  serves  for  both  Incoming  and 
Outgoing  Information  Handling.  An  example  of  one  of  these  is  given  in  Fig.  3.  Some  of  the 
work  done  here  could  logically  be  grouped  with  the  subsequent  detailed  analysis  step;  the 
practical  distinction  between  the  two  is  that  the  data  reduction  and  collation  draws  from  the 
hastily  scrawled  notes  of  several  observers  at  different  physical  locations  and  different  times, 
while  the  detailed  data  analysis  draws  primarily  from  the  intermediate  data  forms. 
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OBSERVER  JOURNAL 


Observer: 


DTAC  0  DMAIN  0  DREAR  0  Brigada  _  Data  -time- 


group_ 


Currant  Ops  Plans  Q  Command  QlntelligencQ]  CSS  Fire  Support]  |  Special  Staf^  | 
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Current  Ops  □  Plans  □  Command  Q  Intelligence  Q  CSS  Q  Fire  Support  Q  Special  Staff  Q 


Stimulus  for  decision: _ w 

T7me  stimulus  perceived  at  CP: _ 

What  was  the  decision? _ 

What  time  was  the  decision  made? 

What  officer  made  the  decision? 

Commander  fl 
Assistant  0  (vision  Commander  Q 

Chief  of  Staff  or  Executive  Officer  O 
G-1/S-1  □  G-2/S-2  □  G-3/S-3  □  G-4/S-4  n 
Suborcinate  in  commanders  name  D 
Other _  Unknown  O 

What  unit(s)  were  affected? _ 

What  elements  did  the  decision  concern? 

Mission  Q  Task  Organization 

Other _ 

Identification  of  directive: 

Type  of  operation  (as  per  attached  list): _ 

Was  a  contingency  activated?  Yes  □ 

If  Yes,  tide  or  other  ID  of  the  contingency: 

Other  related  data  sheets: _ 

Figure  3.  Decision  Context  Data  Reduction  Form 


n  Schedule  Q  Boundaries  Q 
_  Unknown  □ 
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The  key  ACCES  concept  which  guides  the  data  reduction  and  collation  is  the  notion  of  a  CYCLE 
variously  referred  to  as  a  C2  cycle,  planning  cycle,  or  decision  cycle.  As  discussed  on  pages 
5-8  above,  the  overall  effectiveness  of  a  C2  system  can  be  described  in  terms  of  the  extent  to 
which  the  plans  generated  by  that  system  are  robust.  If  the  plans  require  only  minor  “course 
corrections”,  then  the  staff  may  be  said  to  be  more  effective  than  if  major  adjustments  to 
mission,  organization,  assets,  schedule,  or  boundaries  are  required.  Many  staff  elements  are 
simultaneously  engaged  in  many  of  the  activities  which  make  up  the  C2  cycle;  i.e.,  handling 
incoming  information,  tracking  the  situation  (assessing  the  current  situation  and  forecasting 
possible  futures),  planning  and  coordinating,  and  processing  directives  (see  Figure  1,  page  7). 
There  are  even  cases  where  decisions  are  being  made  simultaneously  in  different  locations.  To 
make  sense  of  this  very  complex  set  of  activities,  we  work  from  decisions  backward  and 
forward  through  the  data. 

For  example,  assume  that  one  of  the  data  collectors  has  noted  that,  at  a  given  time,  the 
Assistant  Division  Commander  for  Maneuver  (ADC(M))  told  a  G3  staff  officer,  "I  had  hoped 
we  would  have  more  time,  but  it  is  clear  that  the  covering  force  will  have  to  withdraw  before 
the  1st  Brigade  is  in  position.  Tell  them  to  begin  withdrawing  within  30  minutes  and  see  if  we 
can  get  some  more  artillery  in  there  to  give  1st  Brigade  more  time.”  During  data  reduction  and 
collation,  these  decisions  are  noted,  and  that  observer's  notes,  and  any  others  which  are  relevant, 
are  scanned  to  determine  the  circumstances  of  the  decisions.  What  information  did  the 
ADC(M)  have  available  to  him?  Was  it  accurate,  timely,  complete?  What  interpretations  or 
understandings  had  been  stated?  What  predictions  had  been  made?  The  data  are  also  scanned 
to  see  the  consequences  of  the  decision.  How  long  did  it  take  before  an  order  was  issued?  Did 
the  order  match  the  decision?  Was  the  order  unambiguous?  Or  did  subordinate  commanders 
request  clarification?  If  clarification  was  requested,  how  long  a  delay  ensued  before  the 
clarification  was  provided?  Were  the  subordinate  commands  allowed  enough  lead  time  between 
issuance  of  the  order  and  the  scheduled  execution  time  for  them  to  do  their  own  planning  and 
preparation? 

In  addition  to  the  data  reduction  based  on  C2  cycles,  some  data  reduction  is  based  on  time 
intervals.  Periodically,  at  pre-scheduled  times,  all  data  collectors  will  have  noted  the  location 
and  status  data  on  critical  units  as  posted  within  the  CP  they  are  observing  This  “monitoring” 
data  is  also  transferred  to  one  of  the  intermediate  forms  for  later  comparison  against  ground- 
truth  data. 

Depending  on  the  number  of  ACCES  personnel  available,  the  intensity  of  the  exercise,  and  the 
exercise  schedule  (8  vs.  12  vs.  24  hour  days),  some  data  reduction  may  take  place  during  the 
exercise.  Typically,  however,  thorough  data  reduction  requires  all  data  collectors  to  work 
together  for  two  days  following  the  exercise. 

The  second  step  of  the  data  analysis  is  the  more  difficult  and  time  consuming.  During  this 
phase  two  or  three  experienced  analysts  work  together  to  develop  the  necessary  quantitative 
ACCES  measures  from  the  information  on  the  data-reduction  sheets.  For  example,  a  judgement 
is  made  for  each  decision  as  to  whether  it  involved  changes  to  an  existing  plan  or  activation 
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of  a  contingency  within  that  plan.  The  information  available  to  the  decision  maker  is  compared 
to  ground  truth  to  determine  whether  he  had  an  accurate,  timely,  and  complete  picture  of  the 
battlefield  when  he  made  the  decision.  Percentages  and  averages  are  computed,  and  tables, 
charts,  and  graphs  are  prepared. 

The  following  tables  (9-11  and  13-15)  taken  from  Quantum  reports  on  ACCES  applications  91- 
1  (Castro,  Collingwood,  and  Ervin;  1991),  91-2  (Castro,  Hicks,  and  Ervin;  1991),  and  91-3 
(Gould,  Collingwood,  and  Ervin;  1991),  illustrate  some  of  the  results  obtained.  For  comparison. 
Table  12  gives  an  aggregation  of  the  Plan  Duration  data  over  nine  exercises  and  Table  16 
shows  Completeness  of  Situation  Assessments  for  those  same  nine  exercises.  Figures  4  and  5 
show  in  graphic  form  the  results  of  Tables  12  and  16. 


Table  9 

Plan  Duration  (Hours)  (Application  91-1) 


DAY 

CP 

1  2 

3 

4  5 

4 

AGGREGATE 

DMAIN 

14.0  3.5 

3.0 

- 

12.0 

Discussion:  Scores  for  this  measure  were  based  on  the  1 3  FRAGOs  issued  by  the  division.  All  FRAGOs 
were  issued  by  DMAIN.  The  median  plan  duration  was  12  hours;  however,  on  Days  2  and  3  four  plans 
had  duration  times  of  less  than  four  hours.  This  was  due  to  mission  and  schedule  changes  necessitated 
by  congestion  on  the  main  supply  routes  (MSRs)  delaying  the  advance  of  division  units.  Median  values  for 
plan  duration  could  not  be  derived  for  Days  4  and  5,  as  a  plan  implemented  on  Day  4  was  still  in  effect  at 
ENDEX,  29.9  hours  later,  as  was  a  Day  5  plan  that  had  been  in  effect  for  12  hours.  Duration  of  the 
division's  plans  reflects  the  battle  activity:  long  duration  plans  in  first  days  during  marshalling  operations, 
short  duration  plans  during  initial  contact,  and  longer  duration  plans  again  during  preparation  for  and 
conduct  of  the  defense. 
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Table  10 

Plan  Duration  (Hours)  (Application  91-21 


DAY 

CP 

1 

2 

3 

4 

5 

AGGREGATE 

DMAIN 

- 

0.8 

3.2 

4.8 

- 

3.8 

DREAR 

- 

6.9 

- 

- 

- 

6.9 

ALL 

- 

3.9 

3.2 

4.8 

- 

4.2 

Discussion:  Scores  for  this  measure  were  based  on  nine  of  the  ten  FRAGOs  issued  by  the  division  after 
STARTEX.  One  plan,  implemented  on  Day  4,  could  not  be  scored,  as  it  was  still  in  effect  at  ENDEX,  after 
27.3  hours.  DMAIN  issued  all  FRAGOs  except  for  one  which  was  issued  by  DREAR.  The  median  plan 
duration,  based  on  the  nine  plans  scored,  for  the  division  was  4.2  hours.  As  indicated  in  [the  table]  on 
Day  2,  one  plan  lasted  less  than  an  hour  due  to  schedule  changes  necessitated  by  congestion  on  the  main 
supply  routes  (MSRs)  that  caused  the  division  to  prioritize  unit  movement  on  the  MSR.  This  schedule 
change  was  implemented  in  the  FRAGO  issued  by  DREAR. 

There  were  no  plans  implemented  on  Day  5,  and  median  value  for  plan  duration  derived  for  Day  4  is  not  a 
true  representative  because  of  the  plan  that  was  still  in  effect  at  ENDEX.  Duration  of  the  division’s  plans 
reflected  the  battle  activity;  short  duration  plans  during  the  offense  and  longer  duration  plans  during 
preparation  for  and  conduct  of  the  defense. 


Table  1 1 

Plan  Duration  (Hours) 

(Aoolication  91-3) 

DAY 

CP 

1 

2 

3 

4 

5 

AGGREGATE 

DMAIN 

6.2 

3.4 

12.7 

5.0 

- 

5.5 

DTAC 

- 

- 

4.5 

- 

- 

4.5 

DREAR 

- 

4.0 

4.5 

2.8 

- 

4.0 

ALL 

6.2 

4.0 

6.4 

4.0 

- 

4.5 

Discussion:  The  division  issued  33  FRAGOs  for  this  exercise,  seven  of  which  were  issued  prior  to 
STARTEX  and  not  scored.  Of  the  remaining  26  FRAGOs,  five  were  administrative  directives  (e.g.,  division 
CSM  setting  up  division  training  program)  and  one  still  in  effect  at  ENDEX  could  not  be  evaluated  for 
duration.  The  median  duration  of  the  plans  implemented  by  the  20  FRAGOs  scored  was  4.5  hours.  The 
short  duration  of  plans  was  driven  by  numerous  task  organization  and  mission  changes  designed  to 
maintain  offensive  momentum  and  control  of  the  MSRs  (traffic,  refugees,  and  enemy  interference).  The 
division  was  compelled  to  change  task  organizations  on  1 2  different  occasions  during  the  exercise 
because  of  unexpectedly  strong  enemy  reaction. 
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Median  in  Hours 


Table  12 

Plan  Duration  (Hoursl  (Aggregate  of  9  exercises^ 


DAY 


CP 

1 

2 

3 

4 

5 

AGGREGATE 

DMAIN 

13.4 

3.5 

4.5 

5.3 

5.3 

4.8 

DTAC 

- 

14.3 

4.5 

3.5 

- 

4.0 

DREAR 

- 

4.5 

4.5 

2.8 

- 

4.5 

MNVRBDES 

13.9 

8.5 

6.1 

- 

- 

8.5 

ALL 

13.4 

4.0 

mm 

5.1 

5.3 

4.9 

Discussion:  These  are  medians,  in  hours,  of  Plan  Durations  for  a  total  of  -me  plans  nvor  nin° 
division  exercises. 

Plan  Duration 


9  Division  Exercises 


a  DMAIN  ~  DTAC  o  OREAR  A  MNVRBDE5 

Figure  4 .  Plan  Duration  for  Nine  Division  Exercises 


Table  13 

Completeness  of  FSAs  (%)  (Application  91-1) 


CP 

DAY 

AGGREGATE 

1 

2 

3 

4 

5 

DMAIN 

- 

0[0/4] 

- 

0[0/1] 

- 

0[0/5] 

DTAC 

17[1/6] 

0[1/19] 

11  [1/9] 

0[0/10] 

0[0/3] 

4[2/47] 

DREAR 

- 

0[0/1] 

- 

- 

- 

0[0/1] 

3d  Bde 

0[0/1] 

0[0/1 2] 

0[0/8] 

0[0/5] 

0[0/1] 

0[0/27] 

ALL 

14[1/7] 

0[0/36] 

6[1/17] 

0[0/1 6] 

0[0/4] 

3[2/80] 

Note:  Numbers  outside  the  brackets  are  percentages;  frequency  ratios  of  assessment  elements 
discussed  to  total  elements  are  in  the  brackets. 

Discussion:  Staffs  at  all  levels  prepared  incomplete  assessments  of  the  friendly  situation. 
Discussion  of  combat  service  support  was  missing  more  than  80%  of  the  time,  and  discussion 
of  task  organization  was  missing  more  than  60%  of  the  time.  During  formal  briefings  the  FSAs 
included  only  unit  activities.  Incomplete  FSAs  led  to  a  misunderstanding  on  Day  1  of  the  status 
of  fuel  at  the  refuel-on-the-move  (ROM)  sites,  which  held  up  progress  in  movement  of  a 
brigade.  Missing  CSS  elements  in  FSAs  necessitated  several  “quick  looks"  to  identify  possible 
shortages  of  FASCAM  and  artillery  ammunition. 


Table  14 

Completeness  of  FSAs  (%)  (Application  91-2) 


DAY 


CP 

1 

2 

3 

4 

5 

AGGREGATE 

DMAIN 

50  [1/2] 

40  [2/5] 

0  [0/3] 

33  [1/3] 

0  [0/2] 

27  [4/15] 

DTAC 

0  [0/5] 

33  [1/3] 

33  [1/3] 

0  [0/2] 

0  [0/1] 

14  [2/14] 

DREAR 

- 

50  [1/2] 

100  [1/1] 

- 

- 

67  [2/3] 

2d  Bde 

- 

0  [0/2] 

- 

- 

- 

0  [0/2] 

ALL 

11  [1/9] 

40  [4/10] 

29  [2/7] 

20  [1/5] 

0  [0/3] 

24  [8/34] 

Note:  Numbers  outside  the  brackets  are  percentages;  frequency  ratios  of  assessment  elements 
discussed  to  total  elements  are  in  the  brackets. 

Discussion:  Staffs  throughout  the  division  prepared  incomplete  assessments  of  the  friendly  situation. 
Discussion  of  combat  service  support  was  missing  more  than  60%  of  the  time,  and  discussion  of  task 
organization  was  missing  more  than  45%  of  the  time.  Some  incomplete  assessments  led  to  confusion 
regarding  which  units  were  in  division  reserve  for  the  attack  phase  on  Day  2  and  led  to  doubt  as  to  the 
adequacy  of  combat  power  in  conducting  a  river  crossing. 
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Table  15 

Completeness  of  FSAs  (%)  (Application  91-31 


CP 

DAY 

AGGREGATE 

1 

2 

3 

4 

5 

DMAIN 

- 

25  [1/4] 

13  [1/8] 

0  [0/4] 

0  [0/9] 

8  [2/25] 

DTAC 

0  [0/2] 

33  [2/6] 

100  [2/2] 

60  [3/5] 

- 

47  [7/15] 

2d  Bde 

100  [1/1] 

100  [1/1] 

- 

- 

100  [1/1] 

100  [3/3] 

ALL 

33  [1/3] 

36  [4/11] 

30  [3/10] 

33  [3/9] 

10  [1/10] 

28  [12/43] 

Note:  Numbers  outside  the  brackets  are  percentages;  frequency  ratios  of  assessment  elements  discussed 
to  total  elements  are  in  the  brackets. 

Discussion:  Staffs  at  all  levels  prepared  incomplete  assessments  of  the  friendly  situation.  Division 
planners,  in  reacting  to  a  succession  of  failed  plans,  began  using  only  the  most  readily  available  friendly 
situation  information,  usually  friendly  activity  and  status,  in  their  haste  to  publish  yet  another  plan.  Thus  a 
series  of  planning  cycles  resulted  in  the  publication  of  plans  which  failed,  in  part,  due  to  incomplete  FSAs. 
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Table  16 

Completeness  of  FSAs  (%)  (Aggregate  of  9  exercises) 


DAY 


CP 

1 

2 

3 

4 

5 

AGGREGATE 

DMAIN 

52(69/132] 

471169/360] 

47(148/318] 

72(56/78] 

38(27/72] 

49(469/960] 

DTAC 

67(80/120]] 

60(115/192] 

69(58/84] 

58(66/114] 

46(11/24] 

62(330/534] 

DREAR 

- 

46(11/24] 

89(16/18] 

- 

- 

64(27/42] 

MNVRBDES 

55[72/132] 

42(107/252] 

31(51/162] 

33(28/84] 

92[11/12J 

42(269/642] 

AVNBDE 

• 

25r3/121 

■KM 

17T1/61 

• 

2115/241 

ALL 

581221/384] 

481405/8401 

471274/5881 

54f1 51/2821 

45149/1081 

5111100/22021 

Note:  Numbers  outside  the  brackets  are  percentages;  frequency  ratios  of  assessment  elements  ni^re^d  to  total 
elements  are  in  the  brackets. 

Discussion:  Staffs  at  all  levels  prepared  incomplete  assessments  of  the  friendly  situation.  Mission  was  discussed 
55%  of  the  time,  task  organization:  41%,  disposition:  55%,  unit  activities:  70%,  unit  status:  50%,  and  combat 
service  support  was  discussed  32%  of  the  time. _ 


Completeness  of  FSAs 


9  0 i vision  Exercises 


□ay  of  Exercise 

Q  DMAIN  *  D.TAC  »  DREAR  A  MNVRBDES 

Figure  5.  Completeness  of  Friendly  Situation  Assessments 
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Planning  Cycle  Time  is  the  time  from  the  first  recognition  of  the  need  for  a  change  in 
mission,  assets,  boundaries,  and/or  schedules  to  the  time  the  directive  making  the  change 
is  issued.  Figure  6  shows  median  Planning  Cycle  Times  for  nine  division  exercises  in  which 
the  cycle  times  for  125  directives  were  recorded.  Thirty-four  of  these  times  were  effectively 

Planning  Cycle  Time 


Day  of  Exercise 

a  OMAIN  DTAC  0  DREAR  A  MNVRBDES  X  AVNBOE 

Jigurs  6.  Planning  Cycle  Time 


zero,  that  is,  the  directive  was  issued  immediately  upon  recognition  of  the  need.  The 
medians  reported  here  are  of  only  the  non-zero  values.  The  grand  median  of  non-zero 
times  for  all  CPs  over  all  days  was  2.3  hours.  Eighty  directives  were  timed  at  DMAIN 
CPs;  sixty-two  of  these  were  measurably  greater  than  zero  and  had  a  median  of  3.6. 
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Percent  Complete 


Measures  1 3.0  and!  4.0  are  combined  in  Figure  7,  which  is  here  called  Completeness  of  Incoming 
Reports.  SITREP  and  INTSUM  elements  are  combined  in  this  figure  and  the  percent  reported  out 
of  the  total  number  of- elements  that  could  have  been  reported  is  given  for  each  CP  and  each 
exercise  day.  Combining  over  CPs  and  Days,  82%  of  the  total  of  432  elements  were  reported. 

Completeness  of  Incoming  Reports 


9  Division  Exercises 


Day  of  Exercise 

O  DMAIN  +  DTAC  o  DREAR  A  MNVRBDES 

Figure  7.  Completeness  of  Incoming  Reports 
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I 


J 


Figure  8  shows  the  combined  accuracy  of  SiTREPs  and  INTSUMs  for  non-location  elements. 
DREAR,  at  58%,  is  somewhat  below  the  other  three  CPs  but  that  figure  is  based  upon  only  19 
elements  out  of  a  total  of  3 10  for  all  CPs. 


Accuracy  of  Incoming  Information 


9  Division  Exercises 


Day  of  Exercise 

°  OMAIN  +  DTAC  <>  DREAR  A  MNVRBOES 

Figure  8.  Accuracy  of  Incoming  Information 


Report  preparation.  Once  the  data  have  been  analyzed  and  formatted  into  summary  form,  a 
report  is  written  which  provides  an  assessment  of  the  C2  processes  observed  during  the  exercise. 
This  stage  requires  considerable  expertise  if  the  report  is  to  be  of  any  value  to  the  unit  or  to  an 
outside  agency  such  as  OPTEC.  A  finding  that  the  TAC  CP  had  accurate  information  (as  defined 
by  the  unit's  SOP)  on  the  location  of  critical  enemy  units  only  70%  of  the  time  is  practically 
meaningless  without  some  context  information.  How  does  this  compare  with  other  similar  units 
which  have  been  observed?  Did  the  data  collectors  note  any  problems  in  the  H^ta  flow  between 
the  G2  and  the  TAC  CP?  Was  an  inexperienced,  unsupervised  soldier  misreading  the  grid 
coordinates  when  he  posted  the  information?  Did  the  ADC(M)  or  a  G3  staff  officer  note  the 
problem  and  attempt  to  obtain  more  accurate  information?  Were  any  minor,  moderate,  or  major 
planning  cycles  initiated  on  the  basis  of  faulty  information? 
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During  our  most  recent  ACCES  applications  (four  in  1991,  four  in  1992,  and  one  in  1993)  we 
focused  a  great  deal  of  attention  on  the  form  and  content  of  ACCES  exercise  reports.  One  of 
the  difficulties  we  encountered  was  balancing  the  two  somewhat  contradictory  goals  of:  a) 
preserving  all  possible  information  about  the  exercise  for  archival  purposes  and  to  facilitate  later 
analyses;  and  b)  providing  succinct,  meaningful  feedback  to  the  host  unit.  As  ACCES  evolved, 
we  tended,  at  first,  toward  collecting  and  reporting  more  and  more  detailed  information. 
Version  93,  described  in  this  report,  reverses  that  trend.  By  identifying  new  “primary” 
measures,  most  of  which  are  aggregations  of  existing  measures,  we  believe  we  have  achieved 
a  more  intelligible  structure  for  reporting  and  explaining  our  data. 


ACCES  Status 

ARI  has  completed  the  planned  development  of  ACCES.  The  second  phase  ARI  effort  de¬ 
scribed  here  had  three  primary  objectives:  a)  bringing  the  measures  into  synchrony  with  tasks 
and  standards  as  described  in  Army  doctrine;  b)  streamlining  the  data  collection  process;  and 
c)  streamlining  the  data  analysis  process.  The  goal  was  to  provide  the  Army  with  a  “turnkey” 
measurement  system  which  could  be  used  easily  and  effectively  by  a  unit  conducting  a  CPX, 
by  a  Test  and  Evaluation  agency,  or  by  researchers  concerned  with  specific  aspects  of  command 
and  control.  A  parallel  project,  being  conducted  in  cooperation  with  the  Center  for  Army 
Lessons  Learned,  is  developing  a  C2  performance  database  for  BCTP  Warfighter  exercises 
which  will  allow  analysts  to  identify  communalities  in  unit  performance  through  examination 
of  ACCES  and  other  data  collected  during  several  exercises. 
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